Using grouping strategy and pattern discovery for delta extraction in a limited collaborative environment

It is very common that the large and multinational business organizations distribute their operational database systems over the wide area networks. The data integration from the distributed and highly autonomous operational database systems has to overcome many organizational and technical problems. Materialized view is the permanently stored relational table that contains integrated data from other data sources. Materialized view is an important technique applied to the implementation of data warehousing approach for data integration. In this approach, the data warehouse system extracts, transforms and loads data from heterogeneous sources into a single materialized view, so data residing in different sources are combined into a unified view. The data sources change over times thereafter materialized view must reflect these changes and make the data up to date. These changes are referred to as delta and the process is called view maintenance. In a distributed environment, view maintenance by recomputing view definition can incur an enormous overhead associated with data transporting and loading. To address this issue, many incremental algorithms for view maintenance have been proposed to compute changes to a view only in response to changes to the data sources, in which it generally assumed that the delta somehow was made ready for use and therefore, the majority of research works have focused on how the data changes at the source systems can be integrated into materialized view. However, the problem of delta extraction from highly autonomous systems in a distributed environment is not a straight forward process nor an efficient one. The heterogeneity and distribution of the operational systems contribute to incompatibility and transmission problems. Additionally, a high level of autonomy of the local chapters in a large organization causes the problems of lack of organization-wide collaboration and support. Currently, only little recognition was given to the organizational and technical challenges associated with delta extraction in a distributed environment. This work considers extracting delta in a distributed environment where the collaboration from highly autonomous operational database management systems is limited